Forward feature selection using Residual Mutual Information

In this paper, we propose a hybrid filter/wrapper approach for fast feature selection using the Residual Mutual Information (RMI) between the function approximator output and the remaining features as selection criterion. This approach can handle redundancies in the data as well as the bias of the employed learning machine while keeping the number of required training and evaluation procedures low. In classification experiments, we compare the Residual Mutual Information algorithm with other basic approaches for feature subset selection that use similar selection criteria. The efficiency and effectiveness of our method are demonstrated by the obtained results on UCI datasets.